1. Field of the Invention
The invention relates generally to devices for making presentations in front of audiences and, more specifically, to devices and methods for making presentations for which interaction with the displayed information through direct-pointing is desired or for which verbal interaction with the audience may be anticipated.
2. Background Art
Technology for presenting computer-generated images on large screens has developed to the point where such presentations are commonplace. For example, the software package POWERPOINT, sold by Microsoft Corp., Redmond, Wash., may be used in combination with a so-called ‘beamer’ to generate interactive presentation material and project for viewing by an audience. Often, such presentations are held in rooms not specifically equipped for the purpose of showing presentations, in which case use is made of a portable beamer in combination with a so-called ‘laptop’ computer. Under these circumstances the projection surface may be a wall of the room.
During a presentation it is desirable for the presenter to be able to move freely in front of the audience while retaining the capability to interact with the presentation and point to specific features on the displayed images. It would also be desirable for the presenter to be able to capture verbal comments made by members of the audience so as to amplify and/or play them back to the larger audience.
In general, interaction with a computer is often facilitated by pointing devices such as a ‘mouse’ or a ‘trackball’ that enable manipulation of a so-called ‘cursor’. Traditionally, these devices were physically connected to the computer, thus constraining the freedom-of-movement of the user. More recently, however, this constraint has been removed by the introduction of wireless pointing devices such as the GYROMOUSE, as manufactured by Gyration, Inc.
Broadly speaking, pointing devices may be classified in two categories: a) devices for so-called ‘direct-pointing’ and b) devices for so-called ‘indirect-pointing’. Direct pointing devices are those for which the physical point-of-aim coincides with the item being pointed at, i.e., it lies on the line-of-sight. Direct pointing devices include the so-called ‘laser pointer’ and the human pointing finger. Indirect pointing devices include systems where the object of pointing (e.g., a cursor) bears an indirect relationship to the physical point-of-aim of the pointing device; examples include a mouse and a trackball. It needs no argument that direct-pointing systems are more natural to humans, allowing faster and more accurate pointing actions.
Indirect pointing devices known in the art include the following. U.S. Pat. No. 4,654,648 to Herrington et al. (1987), U.S. Pat. No. 5,339,095 to Redford (1994), U.S. Pat. No. 5,359,348 to Pilcher et al. (1994), U.S. Pat. No. 5,469,193 to Giobbi et al. (1995), U.S. Pat. No. 5,506,605 to Paley (1996), U.S. Pat. No. 5,638,092 to Eng et al. (1997), U.S. Pat. No. 5,734,371 to Kaplan (1998), U.S. Pat. No. 5,883,616 to Koizumi et al. (1999), U.S. Pat. No. 5,898,421 to Quinn (1999), U.S. Pat. No. 5,963,134 to Bowers et al. (1999), U.S. Pat. No. 5,999,167 to Marsh et al. (1999), U.S. Pat. No. 6,069,594 to Barnes et al. (2000), U.S. Pat. No. 6,130,664 to Suzuki (2000), U.S. Pat. No. 6,271,831 to Escobosa et al. (2001), U.S. Pat. No. 6,342,878 to Chevassus et al. (2002), U.S. Pat. No. 6,388,656 to Chae (2002), U.S. Pat. No. 6,411,277 to Shah-Nazaroff (2002), U.S. Pat. No. 6,492,981 Stork et al. (2002), U.S. Pat. No. 6,504,526 to Mauritz (2003), U.S. Pat. No. 6,545,664 to Kim (2003), U.S. Pat. No. 6,567,071 to Curran et al. (2003) and U.S. Patent Application Publication No. 2002/0085097 to Colmenarez et al. (2002). Each of the foregoing publications discloses a system for which the 2 dimensional or 3 dimensional position, orientation and/or motion of an object, such as a handheld pointing device, are measured with respect to some reference coordinate system using appropriate means. Such means include acoustic devices, electromagnetic devices, infrared devices, visible light emitting diode (LED) devices, charge coupled devices (CCD), accelerometer and gyroscopic motion detectors, etc. Although for some of the foregoing devices the reference coordinate system may be positioned close to the display means, no information on the actual position of the presentation display with respect to the system is used, causing the resulting pointing action to be inherently indirect and, hence, less natural to the human operators of such systems.
Other inherently indirect-pointing systems that do not require the position or orientation of the pointing device to be known include devices such as disclosed in U.S. Pat. No. 5,095,302 to McLean et al. (1992) and U.S. Pat. No. 5,668,574 to Jarlance-Huang (1997). The foregoing patents describe indirect-pointing methods, that do not provide the speed and intuitiveness afforded by direct-pointing systems.
Direct pointing devices are disclosed, for example, in U.S. Pat. No. 4,823,170 to Hansen (1989), which describes a direct-pointing system comprising a light source, a position-sensing detector located adjacent to the light source and a focusing reflector that, in one application, is parabolic shaped and is attached to the pointing device. Additionally, procedures are described to calibrate the system. In the understanding of current applicant, however, the physical location of the position-sensing detector needs to be, at least preferably, adjacent to the display means. The system disclosed in the Hansel '170 patent cannot easily be ported to a room not specifically equipped for this system.
U.S. Pat. No. 5,929,444 to Leichner (1999) discloses a system primarily intended for target shooting practice, but an application as a direct-pointing cursor control apparatus may arguably be anticipated. The system includes transmitting and detecting equipment in a fixed reference base and a moveable pointing device. A calibration routine is described that aligns the detecting and transmitting means while keeping the pointing device (i.e., a gun) aimed at the center of the target. The Leichner '444 patent does not describe methods or means that allow determination of a point-of-aim of a pointing device on a target of which the size and/or orientation have not been predetermined. Consequently, the system disclosed in the Leichner '444 patent is not suitable to be used as a cursor control means for projection surfaces not specifically adapted to be used with such system.
U.S. Pat. No. 5,952,996 to Kim et al. (1999), U.S. Pat. No. 6,184,863 to Sibert et al. (2001) and US Patent Application Publication No. 2002/0084980 to White et al. (2002) disclose direct-pointing systems where the 3 dimensional position and/or orientation of the pointing device is measured with respect to sources and/or detectors, the physical position of which in relation to the display means is presumed known. Such systems only work in rooms specifically equipped for their use.
U.S. Pat. No. 5,484,966 to Segen (1996) , U.S. Pat. No. 6,335,723 to Wood et al. (2002) and U.S. Pat. No. 6,507,339 to Tanaka (2003) disclose methods suitable for direct-pointing that are useful only if the pointing device is physically close to or touching the display area or volume, for example used with so-called ‘interactive whiteboards’. Some of the foregoing patents describe appropriate pointer calibration routines. Such systems are not suitable for presentations where the display surface is out of the presenter's physical reach.
U.S. Pat. No. 6,104,380 to Stork et al. (2000) discloses a direct-pointing system in which at least the 3 dimensional orientation of a handheld pointing device is measured by appropriate means. Additionally, a direct measurement is made of the distance between the pointing device and the displayed image. However, the system disclosed in the Stork et al. '380 patent does not include methods to ascertain the position and orientation of the display means relative to the pointing device. In the foregoing system, these appear to be presumed known. This is also the case for a system disclosed in U.S. Pat. No. 4,768,028 to Blackie (1988), in which the orientation of a helmet-mounted direct-pointing apparatus is measured electromagnetically. The foregoing systems therefore appear to be ill-suited to operate in rooms not specifically equipped for presentation purposes.
U.S. Pat. No. 6,373,961 to Richardson et al. (2002) discloses a direct-pointing system using helmet-mounted eye tracking means. The point-of-gaze relative to the helmet is measured as well as the position and orientation of the helmet relative to the display. The latter is accomplished by equipping the helmet either with means to image sources mounted at fixed positions around the display or with means to image a displayed calibration pattern. Of course, the foregoing system relies on sophisticated helmet mounted equipment capable of, among other things, tracking eye-movement. Moreover, such a system relies on an unobstructed line-of-sight with respect to the display and a substantially constant distance from the display to the helmet-mounted equipment. The disclosed invention does not lend itself to be easily used by a human operator in an arbitrary (not predetermined) presentation setting.
U.S. Pat. No. 6,385,331 to Harakawa et al. (2002) discloses a system that uses infrared technology in combination with image recognition software to distinguish pointing gestures made by the presenter, without the need for an artificial pointing device. The disclosed system, however, requires the presentation room to be set up with highly tuned and sophisticated equipment, and is therefore not easily ported to a different venue.
U.S. Pat. No. 6,404,416 to Kahn et al. (2002) discloses a direct-pointing system where a handheld pointing device is equipped with an optical sensor. In such system either the display is required to be of a specific type (e.g., a cathode ray-based display that uses an electron beam) or the displayed image is required to be enhanced by timed and specific emanations. When pointed to the display, a handheld pointing device may detect the electron beam or the timed emanations, and the timing of these detections may then be used to ascertain the point-of-aim. The disclosed system is somewhat similar to the technologies used for so-called light guns in video games as disclosed, for example, in U.S. Pat. No. 6,171,190 to Thanasack et al. (2001) and U.S. Pat. No. 6,545,661 to Goschy et al. (2003). Of course, such systems require either a specific display apparatus or a specialized modification of the displayed images. Moreover, an uncompromised line-of-sight between the pointer and the display is a prerequisite for such systems.
U.S. Patent Application Publication No. 2002/0089489 to Carpenter (2002) discloses a direct-pointing system that, in one embodiment, positions a computer cursor at a light spot projected by a laser-pointer. The system relies on the use of an image-capturing device to compare a captured image with a projected image. As such, the system disclosed in the '489 application publication makes use of calibration routines in which the user is required to highlight computer-generated calibration marks with the laser pointer. The system disclosed in the '489 patent application publication is not unlike a system disclosed in U.S. Pat. No. 5,502,459 to Marshall et al. (1996). Also, U.S. Pat. No. 5,654,741 to Sampsell et al. (1997), U.S. Pat. No. 6,292,171 to Fu et al. (2001), U.S. Patent Application Publication No. 2002/0042699 to Tanaka et al. (2002) and U.S. Patent Application Publication No. 2002/0075386 to Tanaka (2002) all disclose systems that can detect a light-spot using optical means. Such systems specifically generally require the use of computationally expensive image processing technologies. All of these inventions require a projection surface with adequate diffusion properties, as well as some form of optical system with a steady and uncompromised view of the display area. As such, they limit the freedom-of-movement of the presenter and place limitations on the position and optical characteristics of the necessary equipment. Also, in some of these inventions fast and involuntary movement of the user's hand may result in a cursor that does not move smoothly or a cursor that does not perfectly track the light spot, causing possible confusion with the user.
Other pointing systems known in the art may be classified as other than entirely direct-pointing or indirect-pointing systems. Such systems include one disclosed in U.S. Pat. No. 6,417,840 to Daniels (2002), which is combination of a cordless mouse with a laser pointer. Although this system incorporates a direct-pointing device (i.e., the laser pointer), the method used by the system for interacting with the presentation is indirect (i.e., by means of the mouse) and therefore does not afford the fast and more accurate interactive pointing actions provided by some other direct-pointing systems described in some of the foregoing publications.
Another system known in the art that uses both direct and indirect-pointing methods is described in U.S. Pat. No. 6,297,804 to Kashitani (2001). The disclosed system is a system for pointing to real and virtual objects using the same pointing device. In the disclosed system, the pointing means switch between a computer controlled light source (e.g., a laser) and a conventional computer cursor, depending on whether or not the user's intended point-of-aim has crossed a boundary between the real and virtual display (i.e., computer-displayed imagery). Various methods are described to establish these boundaries. Although the computer-controlled light source may be regarded as pointing in a direct manner, its point-of-aim is essentially governed by the system operator using an indirect-pointing system such as a mouse. Thus, the disclosed system does not allow for the desired flexibility afforded by truly direct-pointing methods.
Other systems known in the art include those such as disclosed in U.S. Pat. No. 5,796,386 to Lipscomb et al. (1998), which discloses a calibration procedure to establish the relation between coordinate systems associated with a handheld device and, in one embodiment, a display unit. The system disclosed in the Lipscomb et al. '386 patent may arguably be applicable as a direct-pointing cursor control system. The disclosed calibration routine requires the user to register at least three 3 dimensional positions of the handheld device with a central processing unit. The disclosed system does not appear to include calibration methods for the case where the display unit is out of physical reach of the system operator. The system is thus not practical for use at an arbitrary venue.
U.S. Pat. No. 6,084,556 to Zwern (2000) discloses a head-mounted display unit that displays information governed by the orientation of the apparatus, as measured by a tracking device. This way, the system creates the illusion of a large virtual display that is being looked at by the system operator. Of course, the large display alluded to does not constitute a projected presentation. Also, no methods are disclosed in the Zwern '556 patent to establish the relative position of the head-mounted apparatus with respect to the display.
U.S. Patent Application Publication No. 2002/0079143 to Silverstein et al. (2002) discloses a system in which the position of a moveable display with respect to a physical document is established. The '143 patent application publication describes calibration routines in which the user is required to activate a registration button when the moveable display is over some predetermined position on the physical document. The disclosed system only relates to 2 dimensional applications and, moreover, cannot be used in situations where the interaction region is out of the system operator's physical reach.
U.S. Pat. No. 5,339,095 to Redford (1994) discloses an indirect-pointing system where the pointing device is equipped with non-directional microphone. Also, U.S. Pat. No. 5,631,669 to Stobbs et al. (1997) discloses the inclusion of a non-directional microphone unit in an indirect-pointing device.